Organizations need to understand their data and their business processes—how their data is generated, how it is transformed, where it comes from, and where it goes. Organizations also need to know how their employees access, process and use their data and computer applications, and which data applications and data services their employees use. Failure to monitor data, data applications and data services can result inter alia in loss of data, sensitive data leakage, malicious attacks, uncontrolled spending, redundancy and regulatory compliance gaps.
To this end, organizations deploy discovery and monitoring solutions that track their data flows and discover data applications and data services being used by their employees. Conventional discovery and monitoring solutions use network traffic inspectors to analyze and discover data flows, data application transactions and data service transactions into and out of an organization. Conventional discovery and monitoring solutions apply network analysis to gain visibility to data applications that run on their networks. Conventional discovery and monitoring solutions rely on network inspecting entities including routers, switches, firewalls and proxy gateways.
However, these conventional discovery and monitoring solutions suffer from several drawbacks, stemming from the fact that a data application, data service or data flow that is not visible to an organization's network inspecting entities cannot be discovered and monitored.
With the evolution of cloud-based software-as-a-service (SAAS), employees of organizations are adopting cloud solutions without centralized control and even without knowledge of their organizations. Such behavior is referred to as “shadow IT”, and arises as users sign up on their own initiatives to data services including inter alia data storage, data applications and messaging services, bypassing information technology (IT) processes, controls and approval of the organizations. Background information about shadow IT is available at https://en.wikipedia.org/wiki/Shadow_IT.
Cloud solutions move data to and from the cloud, which is outside of organization networks, and use applications provided by third parties outside of organization networks. Moreover, employees of organizations use these cloud data applications and data services from their homes and from their mobile devices and interact with these cloud data applications and data services over public networks such as the Internet, and generally the data does not flow through the organizations' network inspection entities. Further complicating discovery and monitoring, cloud services use often encrypted network traffic and, as such, even if the traffic is routed through an organization's network inspection entity, the network inspection entity is unable to determine the nature of the traffic and is thus unable to discover shadow-IT traffic. Furthermore, obtaining network logs from network inspection entities such as firewalls, switches, routers, requires inter alia substantial effort and delegation of access rights, which make such network logs difficult to access.
Reference is made to FIG. 1, which is a prior art system for IT discovery. FIG. 1 shows an enterprise network 100 employing a conventional network traffic inspector 110 to inspect data traffic into and out of an organization, and employees 1-5 of the organization. FIG. 1 also shows an Internet cloud 200 and cloud services 210A, 210B and 210C. For example, cloud services 210A, 210B and 210C may include a collaborative document management service, such as OFFICE 365® developed by Microsoft Corporation of Redmond, Wash., a file sharing service, such as BOX.NET® developed by Box.net, Inc. of Palo Alto, Calif., and an e-mail service, such as GMAIL® developed by Google Inc. of Mountain View, Calif.
Employees 1, 2 and 5 are working within the organization or within virtual private networks of the organization, and their data traffic is indeed monitored by network traffic inspector 110. Employees 3 and 4, however, are accessing cloud-based services 210A, 210B and 210C directly from locations outside of the organization and, as such, escape monitoring by network traffic inspector 110.
It would this be of advantage to provide robust discovery and monitoring systems and methods that cover today's cloud/SaaS environments.